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o . 

i Equilibrium statistical physics is considered from the point of view of statistical estimation theory. 

, This involves the notions of statistical model, of estimators, and of exponential family. A useful 

property of the latter is the existence of identities, obtained by taking derivatives of the logarithm 
of the partition sum. It is shown that these identities still exist for models belonging to generalised 
exponential families, in which case they involve escort probability distributions. The percolation 
model serves as an example. A previously known identity is derived. It relates the average number 
of sites belonging to the finite cluster at the origin, the average number of perimeter sites, and the 
CO , derivative of the order parameter. 
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Q . I. INTRODUCTION 

Central to this paper is the notion of escort probability distribution. It was introduced in non-extensive thermo- 
statistics [jj on heuristic grounds. The notion has been generalised in Q, where it was shown that the definition of 
exponential family can be generalised by considering pairs of probability distributions, one of which is called the escort 
of the other. The probability distributions of nonextensive thermostatistics then belong to generalised exponential 
families. 

The example used in the present paper is that of the site percolation problem. Percolation theory is a well- 
established domain of statistical physics 3|. The number of publications in this domain is vast. It is not the intention 
' of the present work to make progress in understanding percolation but rather to show that the notions of escort 
probabilities and of generalised exponential family in a natural way fit in and lead to an interesting result. 

The next two sections relate equilibrium statistical physics to estimation theory and recall well-known material such 
as the definition of the exponential family. In Section 4, the definition of escort probability distribution is given. It is 
used in subsequent sections to recall the generalised lower bound of Cramer and Rao and the definition of generalised 
exponential families. Section 7 shows that for any generalised exponential family one can derive identities, as many 
. as there are model parameters. Sections 8 and 9 treat the example of percolation. The final Section 10 contains a 
' short discussion. 
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In first instance, statistical physics studies statistical properties of model systems. Hence, the mathematical theory 
of parameter estimation can be applied. In this theory, the average value of one or more quantities, called estimators, 
is used to estimate parameters of the model. Here, these quantities are called Hamiltonians because, quite often in 
statistical physics, one of the parameters of the model is inverse temperature (3 and because energy, which is the 
average value of the Hamiltonian, is used to estimate temperature. 

More formally, a model consists of a probability distribution pg, which depends on some parameters 9\, 82, ■ ■ ■ n , 
and a set of Hamiltonians Hi, H2, ■ ■ ■ H n , which can be used to estimate the value of the parameters. In statistical 
physics all these Hamiltonians are added up to form a single Hamiltonian H . This is not done here for simplicity of 
notations. Let us illustrate this point. The Hamiltonian of the d — 1-Ising model is 
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II. PARAMETER ESTIMATION IN STATISTICAL PHYSICS 



H = - J ^ VmCm+l <Jm - W 

m— 1 rn—1 

The variables a rn can take on the values ±1. The probability distribution of the model is 

p(a) = - exp(-j3H) with Z = ^ exp(-/3H). (2) 



The parameters of the model are inverse temperature /3 > and external field h. However, it is convenient to introduce 
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new parameters 6\ = j3 and O2 = f3h, and corresponding Hamiltonians 

n — l n 

Hi(a) = - J ^ er m cr m+ i and #2(0-) = - ^ c%s- (3) 

m— 1 m— 1 

Then, using Einstein's summation convention, the probability distribution can be written as 

p(<r) = ±eM~O k H k (v))- (4) 

III. EXPONENTIAL FAMILY 

A parametrised probability distribution pg{i) belongs to the exponential family if it can be written into the form 

Pe (t) = c(t)e G ^- ekH ^. (5) 
The function G{9) is determined by the normalisation condition and is given by 

G{6) = -ln^c(i)e- e * H *W. ( 6 ) 

i 

In the present paper, the variables are called Hamiltonians. At first sight one might think that every non- 
vanishing probability can be written in exponential form. However, the variables c and Hk in J^J) must not depend on 
the parameters 9 k . This poses a rather strong condition, which is not always satisfied. An example of a distribution 
not belonging to the exponential family is 

11 00 1 

P^^W)^^ With Z{a) = ^a^- (?) 

i— 

Of course, the Ising model belongs to the exponential family with two parameters. 

IV. ESCORT DISTRIBUTIONS 

Given a model with probability distribution pe(i), any other probability distribution Pe{i), depending on the same 
parameters, is an escort distribution for the given pe{i). However, of interest are pairs of distributions pe{i), Pe(i) 
which satisfy some special relation. In 0] the well-known inequality of Cramer and Rao was generalised to pairs of 
probability distributions and a sufficient condition was given that, when satisfied, makes the inequality optimal. The 
usual inequality of Cramer and Rao is optimal in case of a distribution belonging to the exponential family. Hence 
it is natural to say that escort probabilities, optimising the generalised version of the inequality of Cramer and Rao, 
generalise the notion of exponential family. This statement is elaborated in the next section. 

The concept of escort probability distributions is borrowed from the theory of fractals, see Q . It goes back to the 
thermodynamical analysis [5| of multifractals |6(, now twenty years ago. 

Given a probability distribution p(i) which does not depend on any parameters, one can construct a parameter- 
dependent family pg{i) by 

Clearly is p\ — p. A short calculation gives 

d fdG 



with 

G(9) = -In ^2p(i) 9 and H(i) = - lnp(i). (10) 
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One can indeed write 

p e (i) = exp(G{0)-0H(i)). (11) 

This shows that the multifractal model belongs to the exponential family. 

In the present terminology, po is an escort of itself, not of p, as said in £| . But except for this slight change in the 
meaning of the word escort, the present concept generalises that of the multifractal context. Also the thermodynamical 
formalism, developed in the theory of multifractals, coincides with that found in 

V. GENERALISED INEQUALITY OF CRAMER AND RAO 

Here we follow Q, with changes in presentation. Introduce the notations 

(A> fl = and ({A))e=Y, p o(W)- (12) 

i i 

The notion of score variables, used in statistics, is generalised to 
Let Hk be variables for which a function F{9) exists such that 



"' " = w F{9) - ,14! 



Then the generalised inequality of Cramer and Rao, valid for arbitrary u k and v , reads 

U k U l [{{H k Ht)) e - ((H k ))g((H,))e]v m V n ((X m X n ))e > 



The inequality is said to be optimal if equality holds whenever u = v. 

A sufficient condition for optimality is that a function Z{9) exists which is such that 



(15) 



' ((X k Xi)) e = Z{8) [((H k Hi))g = ~(Hi) g . (16) 



^ ^Q~^j W ™ " ' I & \ J L\\ fS "i//P \ \ ■* J -re / / P \ \ J - J -l/ I u \ QQ^ 

An slightly stronger condition is that functions Z(9) > and G(9) exist for which 

pp»(i) = Z(tf)fl,(i)(^-F fc «)). (17) 
See Appendix A. It is the latter condition that is used to define generalised exponential families. 

VI. GENERALISED EXPONENTIAL FAMILIES 

Obvious solutions of (fTTjl are of the form 



Peii) = c(i)fi\G{e)-6*H k {i)\ (18) 

= ^rkdm-o'Hk^)] (19) 



Z{6)- 

Z{9) = Y J ^)n(G{9)-0 k H k {i)), (20) 
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where c(i) is a positive constant and where fi(x) is a positive non-decreasing (stochastic) function. fl(x) is the 
derivative of fi{x) (For convenience, the dependence of fi(x) on the stochastic variable i is written as an index; when 
possible the dependence on i is omitted). The normalisation G(6) must be such that 

<i)fi (g(6) - d k H k (i)\ = 1. (21) 

i 

A function (f>i(y) is defined by 

My) = /l(V(y)), (22) 

with f~ 1 {y) the inverse of the function fi{x). Then one has 

4^ p e(i) = -i^(-FTPe(i) V (23) 



c(i) v y Z(6»)^'Vc(i) 

The only function which is its own derivative is the exponential function. Hence, with fi{x) — exp(x) one finds 
4>{x) = x, Z{&) — 1, and Pg(i) — pe{i)- In this case one recovers a distribution belonging to the exponential family. 

The function f(x), when not stochastic, has been called a (^-exponential function [2, because it satisfies the 
equation f'(x) = <f>(f(x)). The inverse function is then called a ^-logarithm. Models with a probability distribution 
satisfying (|18|l are said to belong to the (^-exponential family. 

VII. IDENTITIES 

A well-known trick of statistical physics is the calculation of averages by taking derivatives of —hiZ. E.g., in the 
Ising model is 

--±aZ = - = (H )e and -_lnZ=- = (^. (24) 

As a consequence, the main problem of equilibrium statistical physics is often the evaluation of the partition sum Z 
to a closed form expression. 

The trick works for all generalised exponential families, and is based on the identity 

Wk - i(H k ))e. (25) 
The latter follows from conservation of probability. Indeed, one calculates 

- ^Epeii) 



f r)C 



It may be disappointing that the quantities that one can calculate via l|25|l are ({Hk))e, and not the physically 
relevant (Hk)e- However, the derivatives of the latter are static susceptibilities, up to a sign, and are given by 

■H k } e = -Z(9)({{H k H l ))g-((H k ))g((H l }}e). (27) 



d9 l 

Hence, the static susceptibilities are controlled by the pair correlations of the escort distribution, showing the physical 
relevance of the latter. Note that the bracket in the r.h.s. of l|27() coincides with the generalised Fisher information 
matrix — see ■ 

The existence of the identities (|25f) and l|27|l makes it attractive to use generalised exponential families. This is 
illustrated below in case of the percolation model. 
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VIII. SITE PERCOLATION 



In the site percolation problem the points of a lattice are occupied with probability p, independent of each other. 
The origin of the lattice is either unoccupied, with probability py, or it belongs to a cluster of occupied sites. This 
cluster is finite with probability one if the density p is less than the percolation threshold p c . Two clusters of the 
same size can have a different shape (these shapes are called lattice animals). The probability that the origin belongs 
to a cluster of shape i is denoted p(i) and is given by 

p(z) = C (z)p s «(l -?)<«, (28) 

where c(i) is the number of clusters of shape i, s(i) is the number of sites, and t(i) is the number of perimeter sites. 
The latter are unoccupied sites that have at least one site of the cluster as a neighbour. Identify shape with the 
absence of a cluster at the origin. Then (|28[) holds with c(0) = 1, s(0) = and t(0) = 1. 

The probability that the origin belongs to an infinite cluster is denoted p(oo). It vanishes for p < p c and is strictly 
positive for p > p c . It satisfies 

p{oo) +Y^p{%) = 1. (29) 

i 

For simplicity of notation we convene that s(oo) = i(oo) = instead of the more obvious infinite value. 
It is possible to write (|28|) into the form 



p e {i) = c(i) exp l[G{0) - eH(i)](s(i) + t{i))\ (30) 

with parameter 9 defined by 

= ln-?-, (31) 



with Hamiltonian 



and with normalisation 



H(i)= (32) 
s(i) + t(i) 



G{9) = -ln(l + e" e ) = \np. (33) 
This suggests the introduction of a stochastic function / defined by 

fi(G) = exp((s(i)+t(i))G) if i < oo 
/oo(G) = 1 - ]T c(i)fi (g + H(i) ln(e G - 1)] . (34) 

Introduce further conventions c(oo) = 1 and H(oo) = 0. Then the probability distribution is of the form 118f) . Note 
that the derivative of / is given by 



f[{G) = (s(i)+t(i))fi(G) if (35) 

(36) 



1 d 



G=G{9) 



The optimal escort probability distribution is therefore given by 

1 

W) 
1 

1-pdO 



IMi) = Jgj(*(i)+*(i))P»(i) if ^ (37) 
= V L , 1 _ 4pp(o°), (38) 



with appropriate normalisation Z{0), given by 



Z(9) = ( s + t) e + -±-%-p e (oc). (39) 
1 — p 69 
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IX. PERCOLATION IDENTITIES 



The identity one can derive from (|25|l reads 



((H)) e = ^ = l-p. (40) 



Using the definition of the Hamiltonian and of the escort probability one finds 



m)e = W) {t)e ' (41) 

Combining (|39l 1401 1411) the identity becomes 

d 

— p e (oo) = {M) , with M(i)=pt(i)-(l-p)s(i). (42) 

This result is known — see (44a) of |3|]. The variable M acts as an order parameter. Its average value vanishes for 
p < p c . It is non-zero for p > p c and diverges as (p — Pc)^ -1 when p decreases towards p c . 
In the present approach the relevant susceptibility is 



Using l(2"Tjl this becomes 



X = Z{e)\J(H')) e -((H))j 
= (HM)g 

= (-L-M 2 )g + (l-p)(M) e . (44) 



It is expected to diverge as \p — p c \ 7 when p approaches the percolation threshold. 



X. DISCUSSION 



In non-extensive thermostatistics average energy is often calculated using the escort probability distribution. Next, 
entropy is maximised under the constraint that the escort averaged energy ((H)) has some given value U — see 
The main point of the present paper is that ((H)) and ((H 2 )) can be used as part of the identities (|25l 1271) . They 
provide a convenient way of obtaining relevant information about the model under study. This approach has been 
illustrated by taking site percolation as an example. 

The presentation has been purely classical, avoiding quantum statistics. A first attempt to treat the quantum case 
is found in The probability distribution and its escort should be replaced by a pair of density operators p and a. 
However, the possibility that p and a do not commute prevents a straightforward translation of the classical formalism 
to the quantum context. 



APPENDIX A: OPTIMALITY OF AND IMPLICATION OF BY iJTTJl 



Let us first show that implies optimality of inequality . From follows 



W {Hl)s = W {Hk) " (A1) 



Hence, a function F(9) exists such that 
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Using l|16f) the inequality then becomes 



i d 2 F 



k L w A m n 
U U „„, V V 



d6 k d8 l 



d 2 F 
ggmggn 



> 



k I 
U V 



d 2 F 



-i 2 



d8 k de l 



(A3) 



Obviously, this is an equality when u — v. Hence, the inequality is satisfied optimally. 
Finally, assume l|17|l holds and prove Condition l|17fl implies that 

d 

% 

f hC 



(A4) 



and 



-Z(9) (H k -((H k )} e ). 



(A5) 



Hence, the first equality of follows. On the other hand is 







= -Z(0)[((H k Hi)) 9 - ((H k )) e m))g] 



(A6) 



This proves the remaining equality. 
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